CROSS-LINGUAL TRANSFER LEARNING

Cross-lingual Transfer Learning for Japanese Named Entity Recognition (N19-2)

Challenge:	XLM-R models encode language-sensitive information in each language, allowing them to extract features for downstream tasks and cross-lingual transfer learning.
Approach:	They evaluate how multilingual language models maintain a shared multilingual representation space while still encoding language-sensitive information in each language.
Outcome:	The proposed model can extract features for downstream tasks and cross-lingual transfer learning.

Copied to clipboard

Andrew Johnson, Penny Karanasou, Judith Gaspers, Dietrich Klakow

MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning (2021.naacl-main)

Challenge:	a recent study focuses on bootstrapping named entity models from English to Japanese . TL is a technique that overcomes linguistic differences between the target and source languages .
Approach:	They propose to use a deep neural network model to transfer weights between languages . they also propose a novel approach that romanizes a portion of the Japanese input .
Outcome:	The proposed approach overcomes linguistic differences by romanizing a portion of the Japanese input.

Copied to clipboard

Mengzhou Xia, Guoqing Zheng, Subhabrata Mukherjee, Milad Shokouhi, Graham Neubig, Ahmed Hassan Awadallah

Bridging the Gap between Native Text and Translated Text through Adversarial Learning: A Case Study on Cross-Lingual Event Extraction (2023.findings-eacl)

Challenge:	Recent work shows that multilingual representations are disjointed across languages, bringing additional challenges for transfer onto extremely low-resource languages.
Approach:	They propose a meta-learning based framework that learns to transform representations judiciously from auxiliary languages to a target one and brings their representation spaces closer for effective transfer.
Outcome:	The proposed framework learns to transform representations from auxiliary languages to a target language and brings their representation spaces closer for effective transfer.

Copied to clipboard

Pengfei Yu, Jonathan May, Heng Ji

Multi-task Learning for Multilingual Neural Machine Translation (2020.emnlp-main)

Challenge:	Recent research in cross-lingual learning has found that combining large-scale pretrained multilingual language models with machine translation can yield good performance.
Approach:	They propose a model architecture that jointly encodes a source language input sentence with its translation to the target language during training and takes a target language sentence with it as input during evaluation.
Outcome:	The proposed model architecture can integrate machine translation to improve event extraction while adding machine-translated data yields unstable performance due to representational gap.

Copied to clipboard

Yiren Wang, ChengXiang Zhai, Hany Hassan

Progressive Sentiment Analysis for Code-Switched Text Data (2022.findings-emnlp)

Challenge:	Existing multilingual neural machine translation systems rely on bitext training data, which is limited and costly to collect.
Approach:	They propose a multi-task learning framework that trains the model with the translation task on bitext data and two denoising tasks on monolingual data.
Outcome:	The proposed framework outperforms pre-training models for both NMT and cross-lingual transfer learning NLU tasks.

Copied to clipboard

Sudhanshu Ranjan, Dheeraj Mekala, Jingbo Shang

Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking (2021.emnlp-main)

Challenge:	Multilingual transformer language models are used in cross-lingual transfer learning for many NLP tasks such as text classification and named entity recognition.
Approach:	They propose a framework that takes the distinction between resource-rich and low-resource language into account and progressively trains from resource-dominated to low-rsource samples.
Outcome:	The proposed model outperforms existing methods on low-resource languages and performs well on medium-resourced to high-res languages.

Copied to clipboard

Nikita Moghe, Mark Steedman, Alexandra Birch

Natural Language Interfaces

Challenge:	Existing methods to make multilingual systems expensive and tedious introduce pipeline of errors.
Approach:	They propose to use pre-trained multilingual models to enhance the transfer learning process by intermediate fine-tuning of pretrained multi-lingual models.
Outcome:	The proposed approach improves on the cross-lingual dialogue state tracking task with only 10% of the target language task data and zero-shot setup respectively.

Retrieving Relevant Context to Align Representations for Cross-lingual Event Detection (2023.findings-acl)

Copied to clipboard

Chien Nguyen, Linh Ngo, Thien Nguyen

Information Retrieval

Challenge:	Existing approaches to cross-lingual transfer learning for event detection are mixed with event-discriminative context.
Approach:	They propose a method where representations are augmented with additional context to bridge the gap between languages while enriching contextual information to facilitate ED.
Outcome:	The proposed model performs well on three languages.

Cross-lingual Transfer Learning with Data Selection for Large-Scale Spoken Language Understanding (D19-1)

Copied to clipboard

Quynh Do, Judith Gaspers

Cross-Lingual Transfer

Challenge:	Existing approaches to improve cross-lingual transfer learning on spoken language are pre-train on all available supervised data from another language.
Approach:	They propose a language model based source-language data selection method for cross-lingual transfer learning in spoken language understanding.
Outcome:	The proposed method reduces training time and improves model performance on spoken language understanding.

Investigating Multilingual NMT Representations at Scale (D19-1)

Copied to clipboard

Sneha Kudugunta, Ankur Bapna, Isaac Caswell, Orhan Firat

Cross-Cultural Similarity Features for Cross-Lingual Transfer Learning of Pragmatically Motivated Tasks (2021.eacl-main)

Challenge:	Multilingual Neural Machine Translation models have shown success in transfer learning settings, but their mode of transfer remains elusive.
Approach:	They propose to use a representation similarity framework to compare multilingual representations using a SVCCA representation similar to the previous work.
Outcome:	The proposed model can be used to compare representations across languages and layers.

Copied to clipboard

Jimin Sun, Hwijeen Ahn, Chan Young Park, Yulia Tsvetkov, David R. Mortensen

MECI: A Multilingual Dataset for Event Causality Identification (2022.coling-1)

Challenge:	a large amount of work on cross-lingual transfer learning focused on typological and genealogical similarities between languages.
Approach:	They propose three features that capture cross-cultural similarities that manifest in linguistic patterns and quantify distinct aspects of language pragmatics.
Outcome:	The proposed features capture cross-cultural similarities manifest in linguistic patterns and quantify aspects of language pragmatics.

Copied to clipboard

Viet Dac Lai, Amir Pouran Ben Veyseh, Minh Van Nguyen, Franck Dernoncourt, Thien Huu Nguyen

Multi-View Cross-Lingual Structured Prediction with Minimum Supervision (2021.acl-long)

Challenge:	Event Causality Identification (ECI) is a task of detecting causal relations between events mentioned in text.
Approach:	They propose a multilingual dataset that provides consistent annotations for event causality relations in five languages.
Outcome:	The proposed dataset provides consistent annotation guidelines for five languages . the dataset can provide ample research challenges and directions for future research .

Copied to clipboard

Zechuan Hu, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu

Multilingual Self-Taught Faithfulness Evaluators (2026.findings-eacl)

Challenge:	Existing work on cross-lingual transfer learning focuses on transferring knowledge from high-resource languages to low-resourced ones.
Approach:	They propose a multi-view framework that integrates multiple source models into an aggregated source view and transfers it to a target view based on a task-specific model.
Outcome:	The proposed framework improves on three structured prediction tasks on 16 datasets.

Copied to clipboard

Carlo Alfano, Aymen Al Marjani, Zeno Jonke, Amin Mantrach, Saab Mansour, Marcello Federico

Pre-Trained Language-Meaning Models for Multilingual Parsing and Generation (2023.findings-acl)

Challenge:	Existing faithfulness evaluation approaches are mostly English-focused and require expensive human-labeled training data for fine-tuning specialized models.
Approach:	They propose a framework that learns exclusively from synthetic multilingual data while leveraging cross-lingual transfer learning to improve an LLM's general language capabilities.
Outcome:	The proposed framework shows that it improves over existing baselines, including state-of-the-art English evaluators and machine translation-based approaches.

Copied to clipboard

Chunliu Wang, Huiyuan Lai, Malvina Nissim, Johan Bos

Syntax-augmented Multilingual BERT for Cross-lingual Transfer (2021.acl-long)

Challenge:	Pre-trained language models (PLMs) have been used for tasks in computational semantics but meaning representations are not included in PLMs.
Approach:	They propose to include meaning representations besides natural language texts in the same model . they propose to use DRSs to improve performance of non-English tasks .
Outcome:	The proposed approach achieves the best performance on multilingual parsing and DRS-to-text generation tasks.

Copied to clipboard

Wasi Ahmad, Haoran Li, Kai-Wei Chang, Yashar Mehdad

Don’t Just Scratch the Surface: Enhancing Word Representations for Korean with Hanja (D19-1)

Challenge:	Existing studies show that pre-trained multilingual text encoders capture language syntax, helping cross-lingual transfer.
Approach:	They provide language syntax and train mBERT to encode universal dependency tree structure.
Outcome:	The proposed model improves cross-lingual transfer on PAWS-X and MLQA benchmarks by 1.4 and 1.6 points on average across all languages.

Copied to clipboard

Kang Min Yoo, Taeuk Kim, Sang-goo Lee

Representation Learning

Challenge:	Existing knowledge of Korean and Chinese is based on cultural and historical reasons.
Approach:	They propose a method for improving Korean word representations using additional linguistic annotation by leveraging the fact that Hanja is closely related to Chinese.
Outcome:	The proposed approach improves representations on a novel Korean news headline generation task.

Cross-Lingual Learning vs. Low-Resource Fine-Tuning: A Case Study with Fact-Checking in Turkish (2024.lrec-main)

Copied to clipboard

Recep Firat Cekinel, Çağrı Çöltekin, Pinar Karagoz

Responsible & Trustworthy NLP

Challenge:	Currently, most of the research on misinformation is focused on the English language . however, there is a scarcity of datasets for other languages, including Turkish .
Approach:	They propose a dataset that spans multiple domains and incorporates evidence from three Turkish fact-checking organizations.
Outcome:	The proposed dataset has the potential to advance research in the Turkish language.

Persian Natural Language Inference: A Meta-learning Approach (2022.coling-1)

Copied to clipboard

Heydar Soudani, Mohammad Hassan Mojab, Hamid Beigy

Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning (2022.coling-1)

Challenge:	In general, shared representations are learned separately, either across tasks or across languages.
Approach:	They propose a meta-learning approach for inferring natural language in Persian . they use different task information or other language information to form additional high-quality tasks .
Outcome:	The proposed method outperforms the baseline approach, improving accuracy by roughly six percent.

Copied to clipboard

Jesujoba O. Alabi, David Ifeoluwa Adelani, Marius Mosbach, Dietrich Klakow

Cross-lingual Transfer Learning for Grammatical Error Correction (2020.coling-main)

Challenge:	Multilingual pre-trained language models have shown impressive performance on several downstream tasks for both high-resourced and low-resource languages.
Approach:	They propose to apply multilingual adaptive fine-tuning to 17 most-resourced African languages and three other high-resource languages to encourage cross-lingual transfer learning.
Outcome:	The proposed approach is competitive to LAFT on individual languages while requiring significantly less disk space.

Copied to clipboard

Ikumi Yamashita, Satoru Katsumata, Masahiro Kaneko, Aizhan Imankulova, Mamoru Komachi

Culture Matters in Toxic Language Detection in Persian (2025.acl-long)

Challenge:	Existing studies on English GEC have focused on improving it, but the resources required to train the models are not sufficient.
Approach:	They investigate cross-lingual transfer learning in grammatical error correction tasks . similarities between these languages is a key factor for successfully transferring grammatikal knowledge .
Outcome:	The proposed methods improve accuracy of grammatical error correction tasks in English and Russian, but lack the resources to train models in these languages.

Copied to clipboard

Zahra Bokaei, Walid Magdy, Bonnie Webber

Superlim: A Swedish Language Understanding Evaluation Benchmark (2023.emnlp-main)

Challenge:	Toxic language detection is crucial for creating safer online environments and limiting the spread of harmful content.
Approach:	They compare different methods for toxic language detection in Persian to fine-tune, enrich data, and cross-lingual transfer learning.
Outcome:	The language of a country with cultural similarities to Persian yields better results in transfer learning.

Copied to clipboard

Aleksandrs Berdicevskis, Gerlof Bouma, Robin Kurtz, Felix Morger, Joey Öhman, Yvonne Adesam, Lars Borin, Dana Dannélls, Markus Forsberg, Tim Isbister, Anna Lindahl, Martin Malmsten, Faton Rekathati, Magnus Sahlgren, Elena Volodina, Love Börjeson, Simon Hengchen, Nina Tahmasebi

Challenge:	In this paper, we present a multi-task benchmark for Swedish language models . we address methodological challenges, such as mitigating the Anglocentric bias when creating datasets for a less-resourced language .
Approach:	They propose a multi-task NLP benchmark for Swedish language models . they propose to use superlim to evaluate Swedish language model performance .
Outcome:	The proposed benchmark does not approach ceiling performance on any of the tasks, suggesting it is difficult to implement.

Distilling Efficient Language-Specific Models for Cross-Lingual Transfer (2023.findings-acl)

Copied to clipboard

Alan Ansell, Edoardo Maria Ponti, Anna Korhonen, Ivan Vulić

A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers (D19-1)

Challenge:	Massively multilingual Transformers (MMTs) are widely used for cross-lingual transfer learning.
Approach:	They propose to extract compressed, language-specific models from MMTs which retain the capacity of the original MMT for cross-lingual transfer.
Outcome:	The proposed model outperforms models trained from scratch in zero-shot cross-lingual transfer across benchmarks.

Copied to clipboard

Aditi Chaudhary, Jiateng Xie, Zaid Sheikh, Graham Neubig, Jaime Carbonell

Responsible & Trustworthy NLP

Challenge:	Named entity recognition models rely on large amounts of labeled data, making them challenging to extend to new, lower-resource languages.
Approach:	They propose a method for bootstrapping named entity recognition models in under-resourced languages . they use cross-lingual transfer learning and targeted annotation of only uncertain entities .
Outcome:	The proposed method achieves competitive accuracy with just one-tenth of training data.

A Chinese Corpus for Fine-grained Entity Typing (2020.lrec-1)

Copied to clipboard

Chin Lee, Hongliang Dai, Yangqiu Song, Xin Li

Information Extraction & Text Mining

Challenge:	Existing datasets for fine-grained entity typing are limited to English . a corpus of 4,800 mentions is manually labeled with free-form entity types .
Approach:	They propose a Chinese fine-grained entity typing task that uses crowdsourcing . they categorize each mention into 10 general types and use a large tag set to predict open set of types .
Outcome:	The proposed dataset contains 4,800 mentions manually labeled in Chinese . it also categorizes all the fine-grained types into 10 general types .

Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model (D19-1)

Copied to clipboard

Tsung-Yuan Hsu, Chi-Liang Liu, Hung-yi Lee

Translation Artifacts in Cross-lingual Transfer Learning (2020.emnlp-main)

Challenge:	Existing studies on RC datasets in English have limited results due to lack of training data.
Approach:	They systematically explore zero-shot cross-lingual transfer learning on reading comprehension tasks with pre-trained language representation model.
Outcome:	The proposed model performs well on reading comprehension tasks on pre-trained language representation models.

Copied to clipboard

Mikel Artetxe, Gorka Labaka, Eneko Agirre

CUTE: A Multilingual Dataset for Enhancing Cross-Lingual Knowledge Transfer in Low-Resource Languages (2025.coling-main)

Challenge:	Existing cross-lingual transfer learning techniques involve human and machine translations.
Approach:	They propose to use machine translation to translate test set or training set to introduce subtle artifacts that have a notable impact in existing cross-lingual models.
Outcome:	The proposed translation process reduces the lexical overlap between the premise and hypothesis by 4.3 and 2.8 points . the proposed translation-test and zero-shot approaches improve on previous work .

Copied to clipboard

Wenhao Zhuang, Yuan Sun

Improving Cross-lingual Transfer with Contrastive Negative Learning and Self-training (2024.lrec-main)

Challenge:	Existing multilingual models such as XLM-R support only approximately 100-200 languages, leaving nearly 7,000 low-resource languages untapped.
Approach:	They construct and open-source a dataset of four-language corpora obtained through machine translation into Chinese, Uyghur and Tibetan.
Outcome:	The proposed dataset includes two resource-rich languages and two low-resource languages.

Copied to clipboard

Guanlin Li, Xuechen Zhao, Amir Jafari, Wenhao Shao, Reza Farahbakhsh, Noel Crespi

Cross-Lingual Transfer

Challenge:	Recent studies improve cross-lingual transfer learning by better aligning the internal representations within the multilingual model or exploring the information of the target language using self-training.
Approach:	They propose to use negative pairs to align the multilingual model and self-train the model to converge on the obtained clean pseudo-labels.
Outcome:	The proposed method improves upon the baseline models and can serve as a beneficial complement to the alignment-based methods.

BembaSpeech: A Speech Recognition Corpus for the Bemba Language (2022.lrec-1)

Copied to clipboard

Claytone Sikasote, Antonios Anastasopoulos

Multimodality

Challenge:	Existing speech recognition systems for African languages are very low . lack of resources (speech and text) can be attributed to poor quality of speech.
Approach:	They present a preprocessed, ready-to-use automatic speech recognition corpus, BembaSpeech, consisting of 24 hours of read speech in the Bemba language.
Outcome:	The proposed model achieves a word error rate (WER) of 32.91% on the Bemba language . the 1 billion XLS-R parameter model achieve better performance than the monolingual pre-trained English model on the corpus.

Can Cross-Lingual Transferability of Multilingual Transformers Be Activated Without End-Task Data? (2023.findings-acl)

Copied to clipboard

Zewen Chi, Heyan Huang, Xian-Ling Mao

Cross-Lingual Transfer

Challenge:	Existing methods for cross-lingual transfer learning cannot activate cross-linguistic transferability when end-task data are unavailable.
Approach:	They propose a cross-lingual transfer method that disassembles multilingual Transformers into sub-modules and reassembles them to be the multilingual end-task model.
Outcome:	The proposed method activates the cross-lingual transferability of multilingual Transformers without accessing end-task data.

XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation (2021.emnlp-main)

Copied to clipboard

Sebastian Ruder, Noah Constant, Jan Botha, Aditya Siddhant, Orhan Firat, Jinlan Fu, Pengfei Liu, Junjie Hu, Dan Garrette, Graham Neubig, Melvin Johnson

XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations (2023.acl-long)

Challenge:	Recent advances in multilingual natural language processing have improved performance on benchmarks such as XTREME and XGLUE by 13 points . however, improvements have been easier to achieve in some tasks than others .
Approach:	They extend XTREME to XTRAME-R, which includes ten natural language understanding tasks and covers 50 typologically diverse languages.
Outcome:	The proposed framework improves the performance on the XTREME multilingual benchmark by 13 points compared to human-level performance on English transfer learning.

Copied to clipboard

Yusen Zhang, Jun Wang, Zhiguo Wang, Rui Zhang